CDS 6334 - Visual Image Processing

Lecture 8: Region Segmentation

1. Why Segmentation?

Segmentation divides an image into meaningful regions or objects.
Pixels belonging to the same object should be grouped together.
🧠 Segmentation = Partitioning an image into meaningful parts.

2. Objectives of Segmentation

Exam Keyword:
Region Grouping

3. Examples of Segmentation Tasks

Task Description
Shot Detection Group video frames into scenes
Region Segmentation Separate image regions
Object Segmentation Extract object boundaries
Figure-Ground Separation Separate foreground and background

4. Top-Down vs Bottom-Up Segmentation

Approach Idea
Top-Down Pixels belong together because they form the same object
Bottom-Up Pixels belong together because they look similar

5. Gestalt Theory

Gestalt Theory explains how humans naturally group visual elements.
Common Gestalt Laws:
  • Similarity
  • Symmetry
  • Closure
  • Continuity
  • Common Fate
🧠 Humans group things that look alike or move together.

6. Segmentation Goal

The primary goal is to separate an image into coherent objects or regions.
Similar pixels should belong to the same segment.

7. Thresholding Revisited

Thresholding separates pixels into groups using intensity values.
Pixels above and below a threshold become different segments.
Important Method:
Otsu's Thresholding

8. Motivation for Clustering

Clustering automatically discovers groups within image data.
🧠 Clustering finds representative groups without manually selecting thresholds.

9. Clustering Review

Pixels are assigned to the nearest cluster centre.
Goal: Minimize Sum of Squared Distances (SSD).

10. K-Means Clustering

K-Means partitions data into K clusters by minimizing within-cluster variance.
Exam Keyword:
SSD (Sum of Squared Distances)

11. K-Means Algorithm

  1. Select K
  2. Initialize cluster centres
  3. Assign points to nearest centre
  4. Update cluster means
  5. Repeat until convergence
🧠 Assign → Update → Repeat

12. Segmentation as Clustering

Segmentation can be performed by clustering pixels in feature space.
Different feature choices produce different segmentations.

13. Intensity-Based Segmentation

Pixels are grouped according to grayscale intensity values.
Feature Space:
1-D Intensity

14. Colour-Based Segmentation

Pixels are grouped using colour similarity.
Feature Space:
RGB (3-D)

15. Intensity + Position Features

Spatial coordinates can be combined with intensity information.
Separates regions that share similar colours but occur in different locations.

16. Texture-Based Segmentation

Pixels are grouped according to texture descriptors.
Feature Space:
Filter Bank Responses

17. Superpixels

Superpixels merge similar neighbouring pixels into larger units.
Significantly reduces the number of elements for processing.
🧠 Many pixels → Fewer superpixels

18. Images as Graphs

Images can be represented as graphs where pixels are nodes and similarities are edges.
Node = Pixel
Edge = Similarity

19. Graph Cuts

Segmentation is achieved by removing weak connections between regions.
Low similarity edges are good candidates for cuts.

20. Normalized Cuts

A graph-based segmentation algorithm that partitions images using pixel affinities.
Exam Keyword:
Spectral Clustering

21. Felzenszwalb Segmentation

Efficient graph-based segmentation method that merges similar regions.
Steps:
  1. Compute edge weights
  2. Sort weights
  3. Merge similar regions
  4. Repeat until complete
🧠 Small weight = High similarity

22. SLIC Superpixels

Simple Linear Iterative Clustering (SLIC) generates compact superpixels.
Requires only one major parameter: Number of Superpixels.

23. Why Are Superpixels Useful?

24. Deep Learning Segmentation Methods

Method Main Purpose
FCN Semantic Segmentation
Mask R-CNN Instance Segmentation
Vision Transformer (ViT) Transformer-Based Segmentation
SAM Prompt-Based Segmentation

25. Segment Anything Model (SAM)

SAM is a foundation model capable of segmenting objects from prompts.
Evolution:
  • SAM 1 → Promptable segmentation
  • SAM 2 → Video segmentation & tracking
  • SAM 3 → Concept segmentation using text/image prompts
Important Concept:
Zero-Shot Segmentation

26. Final Exam Summary

Most Important Points

  • Segmentation: Divide images into meaningful regions.
  • Gestalt Theory: Human-inspired grouping principles.
  • Thresholding: Basic segmentation using intensity.
  • Clustering: Group pixels based on similarity.
  • K-Means: Assign → Update → Repeat.
  • Feature Spaces: Intensity, colour, position, texture.
  • Superpixels: Merge pixels into larger meaningful units.
  • Graph Cuts: Remove weak connections between regions.
  • Normalized Cuts: Spectral graph segmentation.
  • Felzenszwalb: Efficient graph-based segmentation.
  • SLIC: Popular superpixel algorithm.
  • FCN: Semantic segmentation network.
  • Mask R-CNN: Instance segmentation.
  • ViT Segmenter: Transformer-based segmentation.
  • SAM: Promptable foundation segmentation model.